Skip to content

[WIP] scheduler exploration sort based on abs tkv difference - #971

Draft
yannicks1 wants to merge 1 commit into
torch-spyre:mainfrom
yannicks1:scheduler-exploration-closest-tkv
Draft

[WIP] scheduler exploration sort based on abs tkv difference#971
yannicks1 wants to merge 1 commit into
torch-spyre:mainfrom
yannicks1:scheduler-exploration-closest-tkv

Conversation

@yannicks1

Copy link
Copy Markdown
Collaborator

Do not merge, experimental only

Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>
@yannicks1

Copy link
Copy Markdown
Collaborator Author

bot:bench
NUM_PROMPTS=1000
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=8
ENABLE_PC=0

@github-actions

Copy link
Copy Markdown

👋 Hi! Thank you for contributing.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, run ./format.sh.
Now you are good to go 🚀.

We also recommend installing prek and configuring it to check your code before every local commit.

@yannicks1

Copy link
Copy Markdown
Collaborator Author

bot:bench
NUM_PROMPTS=1000
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=8
ENABLE_PC=0

@yannicks1

Copy link
Copy Markdown
Collaborator Author

bot:bench
NUM_PROMPTS=1000
MAX_RUN_TIME=36000
IGNORE_EOS=1
CUSTOM_OUTPUT_LEN=-1
MAX_CONCURRENT=32
ENABLE_PC=0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant